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GENERAL BACKGROUND ON THE INTERMEDIATE SCIENCE CURRICULUM STUDY 



The Intermediate Science Curriculum Study (ISCS). is a larger-scale 
instructional research project supported to date by a contract with 
the United States Office of Education and grants from the National 
Science Foundation. The project is designed to develop, test, and 
disseminate into practice a system of individualized science instruc- 
tion for grades seven -through nine. 

The project is organized on a develop-field-test-revise design. Draft 
materials are produced at Florida State University by on- and invited off- 
campus personnel and tested on a large national sample of junior- 
high-school students. During the 1969-70 school year, more than 
75,000 students in 25 states are involved in the field testing of 
the ISCS materials. In addition, a small number of students from 
the Florida State University campus school are taking a computer- 
assisted instruction version of the materials from which additional 
feedback data are being accumulated. To date, more than l|00 scient- 
ists, teachers, and education specialists have cooperated in the 
development process. 

The most unique feature of the ISCS materials is the fact that the 
'students using them progress at different rates and through differ- 
ent instructional pathways depending upon their interests, abilities, 
and previous experience. The materials are being designed that this 
can be accomplished in ordinary science classrooms by teachers with 
limited special training. 

The package of instructional materials for each grade level consists 
of student printed materials, specially designed laboratory appara- 
tus, a student self-evaluation system based upon behavioral objec- 
tives established for the instructional materials, teacher orienta- 
tion materials, and standardized tests. The Silver Burdett Corpor- 
ation, in conjunction with Damon Educational Corporation, is dis- 
tributing these materials during the experimental phase of the 
project and will market the commercial version of them. 

The project has generated world-wide interest and its newsletter, 
published twice yearly, now goes to more than 10,000 people in 
countries. ISCS materials are now in use in Australia and will be 
used in American dependent schools in Germany and Japan in 
September. Experimental testing of the materials is now underway 
in Manila, and plans have been established for a joint Florida 
State University - Philippines effort to produce a special 
Philippines version of the program. In addition, project personnel 
have visited Japan, India, and several South American countries 
for preliminary discussions related to possible use of the 
materials in these areas. 
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CAI UTILIZATION FOR FORMATIVE CURRICULUM EVALUATION 



Introduction 



Much effort and substantial funds have been expended in 
developing Computer Assisted Instruction (CAI) as a specialized 
learning mechanism. In using this instructional medium efforts 
have been concentrated on a variety of learning studies involving 
■extensive and ingenious schemes of .presenting information to the 
learner or otherwise involving him in the learning process. Hie 
developing CAI technology is plagued by concerns of hardware expense, 
dearth of analysis and materials, software, and a questionable future 
market for such a modality. 

In 1966, the Intermediate Science Curriculum Study (ISCS) began 

a large scale curriculum development effort in which a specialized 

, * 

CAI application was envisioned as an economically defensible and 

uniquely valuable research and development tool for the curriculum 

developer. This anticipated use of Computer Assisted Instruction 

to assist in the formative evaluation of ISCS materials was evident 

in a then bold proposal by Burlonan . ^ 

Using an adaptation of "coursewriter" techniques, the completed 
curriculum materials will be programmed l or CAI in their en- 
tirety to permit logical, step-by-step presentation to the 
students. Because of the semi- programmed nature of the class- 
‘ room materials to be produced, the task of converting them 
to a programmed sequence will not be as formidable as would 
be the case with conventional materials. In the programming 
process, every effort will be made to reduce to an absolute 
minimum the variation between the CAI and printed versions of 
the course with regard to structure, content, aid wording. 

..^B.u^lqnan,, E. _( Principal^ Investigator) “A Project for the Development 
and Evaluation of a Coordinated Science Curriculum for Grades Seven 
Through Nine." A proposal for Research and/or Related Activities 
submitted to the U.S. Commissioner of Education for Support Through 
Authorizations of the Bureau of Research, Florida State University, 
Dec. 2„ 1965. 1 
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As each student proceeds through the CAI program, his responses 
will be identified and stored in the machine on disc packs or 
cards for later analysis. This will produce an individualized 
record of the specific difficulties which various types of 
students encounter as they pass through the sequence. Such 
information should provide unique advantages in terms of the 
types of revisions needed in the general approach taken in 
the written materials as well as specific adjustments required 
for specific types of students. Hopefully, the analysis of 
the stored CAI performance data will provide: 

1. A detailed breakdown of student performance on each step 
of the curriculum. 

2. Suggestions as to appropriate branches which should be 
built into the curriculum to accommodate various categories 
of students. Such information could be utilized in building 
the supplementary curriculum materials (Excursions), in 
making the existing curriculum more effective, or perhaps 

in building parallel courses for various ability levels. 

3. The step-wise performance data which can be correlated 
specifically to the reading and quant itati/j aptitude measures 
and to the achievement test or its sub-test. 

Burkman had envisioned and proposed the ISCS model for the 

teaching-learning situation where much of the student’s work is 

conducted independently . In eh.-.s approach, the learning pace is 

set by the student, and the level of instruction is automatically 

adjusted to his ability. 

To allow for individual differences in ability, two types of 
student materials have been developed. The primary sequence (core) 
for each grade provides the basic "story line" that every student 
follows. "Excursions' 1 are materials that provide departures from 
the primary sequence. There are two kinds of excursions. Enrich- 
ment excursions are designed to provide greater challenge for the 
more able student, or for the student interested in pursuing a 
topic in greater uep.n. Remedial excursions provide the slower 
student, or less well-prepared student, with special background or 
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skills needed for efficient progress in the core. Through selection 
and judicious use of excursions, the ISCS student or teacher may 
select a multi- track program specifically geared to the needs of 
the individual. 

Both core and excursions are written in a semi-programmed style 
which permits students to proceed independently at their own pace. 
Thus the student text serves as lab guide and record book, as well 
as a source of conceptual and factual information. 

The use of CAI was proposed as potentially the most appropriate 
and efficient means of monitoring in detail each student’s individual 
progress and performance in such material. Witn such a monitor, 
the curriculum developer could hope to pinpoint the exact areas 
of the sequence that cause difficulty for specific types of indivi- 
duals . Information of this sort could make the task of building 
and revising instructional materials easier and much more efficient. 

The plan proposed and, in fact, essentially put into practice 
was to program each of the grade seven through grade nine science 
materials on a grade -a-y ear basis concurrently with the preparation 
of field trial versions of the same material. This allowed barely 
six months to complete the programming at each grade level. During 
this same time period at least half the student performance data 
had to be analyzed and presented to revision teams. 

This report outlines the chronological development of the ISCS 
technology of utilizing CAI in this application. The procedures 
used and their effectiveness are reported as a guide to future re- 
searchers interested in similar applications to other subjects and 
grade levels. 






THE IBM 11(00 SYSTEM - A First Attenpt 
The formative evaluation of ISC'S materials, using CAI, began 
in the fall of 1966. This corresponded to the first year of the 
classroom trial of the text materials. Simultaneously, a presenta- 
tion of the ISCS program was made possible through the cooperative 
assistance of the staff of the Florida State University CAI Center. 

At the time, the CAI Center had an IBM 11(00 computer with eight 
typewriter input- output terminals. With this system, all instruc- 
tion and student responses processed by the computer were typewritten 
messages . 

The problem immediately faced by ISCS was hew this facility 
could be best adapted to satisfy the formative requirement. Wo 
group had ever undertaken so ambitious a task as programming for 
CAI presentation a whole year's course, much less a three-year 
sequential program. Further, formative analysis mechanisms were 
virtually nonexistent at the time. The task faced was formidible, 
particularly in the light of the fact that barely a three-months 
time period was allowed between the beginning of the preparation 
of field trial curriculum materials and the time that operating 

programs were available for CAI presentation to seventh grade subjects. 
Procedures For Programming : 

From the beginning, the chief goal of the ISCS use of CAI has 
not been to produce the best possible computer program for science 
instruction. The closer the match between computer program and 
classroom text the better. The focus, then, was on producing a 
detailed, accurate set of real student data for use in the prepara- 
tion of a sound three-year course for classroom presentation. 



This fact provided the critically needed basis for simplifying the 
task of the programming of student materials. 

The programming language used for the IBM 1400 system was Course 
writer. I. This language and the hardware that utilized it could 
not accommodate illustrations , graphs, or charts. Eecause the in- 
put-output procedure via typewritten messages was relatively slow 
compared to the student’s reading rates, long passages of written 
text were frequently presented via a printed document called the 
student text. Data tables and graphs were also included in the 
student text. There was relatively little art work in the first 
edition materials, and this was provided by means of a series of 
35mm slides to which students were referred by the computer or by 
their text. 

Laboratory activities were done in the immediate vicinity of 
the computer terminal. Fortunately many of the activities did not 
require a large working area. 

One critical question in the minds of the ISCS writers and 
CAI programmers was; Could the students do the coursework with 
the computer acting as teacher with no human direction-giving or 
assistance? The answer was a definite no! The aid of a proctor 
was required to explain some instructions and help develop certain 
critical concepts. The proctors also helped alleviate equipment 
malfunctions, handled equipment storage and inventory problems, 
and served as valuable on-the-spot recorder of observable "classroom" 
problems and computer program "bugs." 



In an attempt to anticipate the questions of revision teams, 
large paragraphs were fragmented with specific questions. By 
increasing the number of questions , the programmers attempted to 
provide a more detailed, smaller-interval tracking of student be- 
haviors for each test section. Declarative statements were frequently 
converted into interrogative ones. For example, suppose the class- 
room text was as follows: 

"Ihe blade of the force measurer is bent by the weight hanging 
on it." The computer program presented this statement as a question: 
"What caused the blade of the force measurer to be bent?" 

A set of multiple-choice responses followed, or the student was 
asked to type a free response answer. 

To make sure the student knew the initial piece of information 
given in the declarative statement, he was then told by the computer 
whether or not his answer was correct-, or he was told what the 
correct answer should be. In this way, each correct student 
response was reinforced. At the time, this programming technique 
seemed most appropriate in that it followed Skinnerian philosophy 
of programmed instruction. The effect these departures had on the 
programing philosophy of duplicating the textual materials was 
to produce substantial change in the CAI material as is shown in 
the sarnple of Chapter 5 duplicated in Appendix A. 

Developing Analyses for Feedback to Rev isi on Teams 

Successful analyses mechanisms rarely materialize without care- 
ful planning at the data-collection stage. So it is with CAI even 
in its first year. Ihe greatest single advantage of the 1*100 system 
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mode of instruction was the permanent typewritten copy of materials 
which could be taken away after each day's instruction. This hard 
copy facilitated review and study by the students (if they were so 
inclined) but also gave a detailed unabridged account of each stu- 
dent's performance to the evaluator. ISC'S had anticipated that the 
programming protocol of increasing the questioning of the student 
would provide rich dividends in the form of revision information. 

A sample of this output is shown in Appendix B. Although only eight 
students were using CAI the first trial year, the volume of this 
output was entirely too overpowering to be digested and used by 
the writers in the limited time available to them. Ihey needed 
and demanded more efficient and pointed information. 

CAI formative evaluation did not come with ready-made data 
analysis programs. Moreover, the variety of computer languages 
and systems being used by CAI groups around the country varied so 
much that efforts to interchange analysis programs were fruitless. 
The FSU-CAI Center was and has remained in the process of developing 
and updating the capabilities of its data management system during 
much of the period of time that ISCS has been using CAI to evaluate 
its curriculum. Consequently, the provision of data for revision 
purposes has required the simultaneous and coordinated efforts of 

ISCS and CAI personnel to evaluate, utilize, and revise the data 
analysis procedures. 

Through this joint effort, the ISCS writers were provided with 
several different types of information during the first year 1 s CAI 
evaluation. For each chapter in the text, the data were summarized 
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several ways. Each student response to each frame of information 
was recorded on magnetic tape along with various identification 
data for that student and frame. An elaborate sequence of data 
analysis programs was developed by CAI personnel to sort, edit, 
and merge the records in order to provide meaningful information 
summaries for the ISCS revision teams . The initial program provided 
a summary for every item which was encountered by at least one stu- 
dent. At the request of the ISCS writers, an alternate, more de- 
tailed program was developed to provide a listing of the first few 
characters of the actual responses of the students to each frame. 

Ihis program also indicated whether each answer by each student 
was a correct answer (ca), wrong answer (wa), or unanticipated 
response (un). More elaborate student responses were recorded in 
printed data tables or on charts and graphs provided on separate 
sheets of paper as supplements to the CAI typewriter presentation. 

Ihe writers also had available a great deal of rather insightful, 
but not unique to CAl, proctor comments recorded on a day-by-day 
basis by the teaching' assistants who worked in the CAI '•classroom" 
Summaries of this information •— conceptual problems identified 
by students, equipment design or structural inadequacies, etc. — 
by chapter were provided. In addition, the ISCS writers were given 
item analyses of conprehensive tests administered after every two 
or three chapters of toxt material. Notes taken during student- 
proctor discussions of the tests we re also given to the writers. 







Lessons Learned Prom First Year 



The CAE data analysis procedures and the utilization of this 
information for revision of the first year version of the ISCS seventh 
grade student text were perhaps as valuable in indicating the possi- 
bilities for future practices and utilization as in aiding the 
revision team which had to produce the revised edition of the text. 
Some of the data collected during the CAI evaluation class in 1966-67 
possibly could have been obtained in a regular classroom. However, 
the unique combination of detailed information on each individual 
student and the "classroom group" types of data obtained during 
the first trial year of CAI were most valuable for ISCS. 

Some of the more important lessons learned for the use of CAI 
with the developing ISCS curriculum were : 

1 . It was possible through the use of Coursewriter programming 
language to completely program, proctor, and analyze the full 
year's course of instruction with two half-time graduate stu- 
dents and CAI Center support personnel under the part-time 
direction of one faculty member. Future programming could be 
done with the same manpower complement or less. 

2 . The revision of first trial materials is so intensive 
because of equipment, logic, and pedagogical considerations 
that programming and testing of first version material is not 
an efficient use of manpower and machine time. The strength 
of CAI for evaluation lies in its complete record-keeping, 
which makes possible a precise evaluation of materials. First 
version text materials — at least the ISCS first version 



materials — do not warrant such a precise evaluation. Most of 
the changes that need to be made are gross and therefore quite 
obvious to a person trained in science or science education. 

CAI evaluation certainly identifies these problem spots, but 
unless there is an abundant supply of manpower, and unless 
computer time is inexpensive, data needed for first revision 
can be obtained more efficiently through regular classroom 
trial. For these reasons, only second version materials for 
any grade level have been evaluated via CAI since the 1966-67 
trial year. Hie original plan to run a simultaneous CAI 
evaluation of the first year eighth-grade along with the second 
year seventh-grade materials was delayed until a year of class- 
room testing and subsequent first revision were completed on 
the eighth-grade materials. 

3. New and more effective analysis routines were still needed 
to pinpoint particular problem areas. Manual data reduction 
of free responses and numerical data reported as student measure- 
ments were inefficient and most difficult to handle. Similarly, 
creative ways of tracking the nature and rate of student pro- 
gress of both the individual and the group were sorely needed. 

In subsequent trial years, substantial technological improve- 
ment was made in these areas. 

L\. Hie decision to provide detailed, small-interval tracking 
of student behavior by increasing the number and specificity 
of questions in the CAI test versions gave unsatisfactory results. 
The substantial way in which these modifications changed the 
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instructional material placed the evaluator in the position 
of not being sure what effect knowledge of results and rein- 
forcement were having on student learning. Therefore, progress 
could not be attributed to text design only. 

Since the first year of CAI evaluation, the general approach 
used for the seventh-, eighth- and ninth-grade materials programmed 
for CAI presentation has been to preserve the exact wording of the 
text and to provide knowledge of results and reinforcement only 
when they are found in the corresponding textbook sections. 

THE IBM 1500 SYSTEM — A New Formative Technology 
Prior to the start of the 19^7-68 school year, the CAI Center 
at FSU installed an IBM 1500 System which provides both typewriter 
and light pen response capabilities along with cathode ray tube 
written and graphic instructional modes. Although ISCS did not 
enter directly into the decision-making procedure by which the CAI 
moved to the new system, ISCS did strongly support the change. 

Using the Coursewriter II language, the programming techniques afforded 
by the IBM 1500 System seemed to greatly increase the potential versa- 
tility of the CAI mode of instruction. Some of the increased 
capabilities of the system included: the use of graphic characters 

and animated figures along with the usual printed matter, increased 
rate and breadth of instruction presentation (now a paragraph of 
printed matter could be presented almost instantaneously at the 
student's request to continue, as opposed to the teletyped messages 
written a line at the time on the IBM 1400 input-output terminals), 
and alternative modalities in the man-machine interface (light pen 
or typewritten, multiple choice or free response). 
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.Software for the Instruc tional P rograms 

Hie decision to match the CAI version of the textual materials 
as closely as possible to the classroom textbooks simplified the 
programming for the computer, A comparative sample of the CAI and 
field trial versions of the materials is shown in Appendix C. Hie 
increased use of a semi-programmed format with activity frames in 
the regular textbooks facilitated the preparation of their contents 
for computer presentation. The three factors — decision to repro- 
duce text materials only, the new semi-programmed format, and the 
flexibility of the new Coursewriter II programming language — re- 
duced by at least one-third the time required to complete the 
materials programming. 

With Coursewriter II ^ there is a limited number of different 
formats required to present the basic kinds of materials found in 
the textbooks. As a result, all the ISO'S materials programmed 
for presentation by the 1500 system were prepared for presentation 
via one of only ten basically different formats, bach of these 
formats is utilized in the form of a macro. 2 

Thus it is possible to program large quantities of textual 
material without the necessity of the programmer's typing each 
detailed statement for each frame of instruction over and over again. 
Only the unique text statements and unique parameters such as 
frame identifiers, response identifiers, and branching labels need 

2 * 

Hie macro is an input programming device available on the 1500 
system which permits the programmer to use a card assembly procedure 
to call a detailed sequence of program statements. These statements 
may have required parameters inserted into designated locations 
by including the parameters in the single ••call macro" statement 
written by the author. 
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to be entered by hand for each frame. Together with these unique 
statements, a single call macro statement is entered for each 
frame, and it may "call in' 1 from *14 to 60 different additional 
statements for conputer storage. 

Obviously, the time and effort required to design, develop, 
and test a macro is justifiable only to the extent that there is 
reasonable utilization of the macro. ISCS has more than adequately 
utilized the macros developed by its staff — in fact, in a real 
sense ISCS has evolved a comprehensive ’’macro based system of text 
input.” 

Ee cause the current version of the ISCS programming techniques 
and procedures represents the culmination and broadest utilization 
of the macros developed over all the three years' efforts of the 
project, the characterization of these macros is given in terms 
of their present state of development. A brief description of each 
macro is given below, and detailed listings of the Coursewriter 
II statements of which these macros are constructed are available 
and listed in the ISCS Tech Memo 1. 

The Macros . Hie ten different types of macros — two of 
them with several different versions available to permit 
versatility such as designating any one of four possible 
answers as the correct one for a multiple choice question — 
are described below. The significance of the use of these 
macros lies in their power to reduce writing and programming 
time and effort. These ten types of macros have served the 
purpose of inputting two versions (1967-68, 1968-69) of the 



m 



grade seven year- long course, one version (1968-69) of the 
grade eight year-long course, and approximately one half (1969-70) 
(3 units) of the grade nine course. Had the macro mode of 
programming not been utilized, the computer language statement 
writing and programming time probably would have been three 



times as great as that actually required. 
Macro Mode* Type of Text Programmed 



ttd500 


LP 


Information Presentation. 


exd500 


LP 


Branch to Excursion decision 
Frame 


ysd500 


LP 


Question which has a yes 
or no answer. Yes is the 
correct response. (A 
Variation provides for no 
as a correct response.) 


mad500 


LP 


Multiple choice questions, 
choice A is correct 
answer. (Variations provide 
for B, C, or D as correct 
answer . ) 


rst500 


LP 


Resource decision frame. 






(Request one of 32 resources 
used in geology unit only . ) 


tst500 


LP 


Test question. (Any one of 
possible answers can be 
programmed as "correct. ’ ) 


dcd500 


KB 


Free response questions. 



Wo specified correct answer. 
(Limited to a one line 
response . ) 



Branches 



Continue 

Review 

Excursion 
next frame 
Review 

Next frame 
(with selection 
of answer) 
Review 



Wext frame 
(via an answer) 
Review 



Continue (to next 
frame) 

Call for resource 
Review 

Skip (to next 
question) 

Review (to previous 
question) 

Respond (to one 
of four 

possible answers). 

Continue (by 
entering a 
response) 

Review 
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cmt500 KB 



lmd500 I© 



kld500 KB 



Free Response— -no specified 
correct answer. May be from 
one to ten lines in length. 



Continue (by 
entering a 
response ) 
Review 



Free response question with Continue (by 

numerical answer. Limit entering an 

function used to determine answer) 

if an answer is within certain Review 
limits and therefore acceptable 



Free response question with 
correct alphanumeric answer. 
Key letter function used to 
determine whether a response 
is acceptable. 



Continue (by 
entering an 
answer) 
Review 



*LP =' light pen; I© = keyboard 



There were also some important changes occurring in the thinking 



of ISCS regarding ways of programming which would better utilize 



the potential of CAI as a powerful evaluative tool. The first year’s 



coding of frames was restricted to identification by chronological 



sequence within the program. This limited classification did not 
facilitate the investigation of the relationships between different 
science content and process themes. The development of a ten- 
character , alphanumeric code made it possible to identify each 
program segment and question according to its position in the text 



sequence as well as identifying the specific science content and 



process with which it deals and its remedial enrichment or diagnostic 
characteristics. It is now possible to retrieve these data in many 
different ways., depending upon particular evaluative interests. 



For example one may be interested in knowing if the difficulty 



0 



of the concept of kinetic energy appears to be related to the 
difficulty with the concept of speed. The coding scheme may be 
used to single out all parts of the text related to these two concepts 
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and investigate the student error rate (individual and/or group) 
for these items. This data bank and retrieval system make possible 
any number of specific investigations that are useful, not only 
in the revision of materials , but in investigating which particular 
content sequence is most logical to different kinds of students. 

The process of coding each frame requires very little extra 
time and was found to be a most valuable asset in the analysis phase data 
reduction. The major exception to this statement is the time re- 
quirement for content and process coding. Although only one character 
was required to accommodate each f unction , several practical problems 
were experienced. A competent science educator was the only person 
who correctly assigned each frame valid code designations, and the 
time he took to do it more than doubled the total programming time 
to produce the CAI materials. Under the pressure of time available 
to get CAI programs operational for student trial, the complete 
coding of all frames had to be given up. As a result, dummy codes 
were used so that valid content and process codes could be added 
later in particular program segments for which revision teams requested 
such analyses. 

Analy ses of I BM 1 5 00 System Data 

Because of the many demands and problems of a first year’s 
effort with the IBM 1^400 system, it was impossible to make the most 
efficient use of the CAI evaluation in the textual revision during 
the summer of 1967. The data analysis provided the revision team 
was fairly complete, but the method of presenting it proved to be 
somewhat inappropriate. Two assumptions were made which later were 



decided to be in error. 
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First., the amount of time writers would have to examine the 
data in some detail before beginning rewrites was overestimated. 
Secondly , it was assumed that the writers should be provided with 
an unbiased presentation of the data., rather than an interpretation 
of what the analysis suggested about the materials. In this way, 
it was thought, the writers would be free to make their own judgments 
without the intervening bias of an interpreter. 

Because time did not allow all the writers to fully digest 
the CAI data, it was apparent that efficient use of the kind of CAI 
data available to ISCS required a greater condensing of the records 
and a fairly specific interpretation of the data analysis. The 
decision was made to present each member of the revision teams a 
summary so that each writer could quickly see its relationship to 
trouble spots in the textual materials and what this relationship 
suggested about needed changes. These summaries were prepared on 
a chapter-by-chapter basis. Individual and group records for 
every student response were provided along with a rate of progress 
chart for each student and the percentage of correct answers for 
each question in the text. 

This type of information has been generated and supplied to 
both the 1968 and 1969 summer revision conferences. The kind of 
information provided in these summaries is illustrated by the 
following paragraph. 



0 
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We find that the concept of kinetic energy is not being 
applied correctly by the student in Chapter 9 as indicated 
by the high error rate on Questions 9-3^ 3 9~37 5 and 9-^1. 

This appears to be related to the difficulty with the concept 
of speed in Chapter 6, Questions 6-12, 6-15, 6-16, etc. It 
may be possible to correct this by rewriting the first activity 
in Chapter 6 , with special attention being given to the 
relationship of time and distance of the moving cart. It 
may also be necessary to write another remedial excursion 
dealing with division of decimals. 

It is apparent that providing concrete suggestions of this sort 

greatly helps the writers in their revision task. The writers have 

stated their preference for this type of feedback format and point 

to a greater facility in incorporating the data provided into the 

total information pool used for revision decisions. 

Several analysis programs were developed to facilitate the 

data digestion and presentation process making possible the type 

summary information provided the revision teams. A description 

of these programs and sample data obtained by their use are presented 

in the remainder of this section of this report. It should be 

noted that the descriptions given here represent the latest and 

therefore the most efficient and useful data analysis package ever 

used by ISC/? for treating the CAI data. 

The basic set of analysis programs available at the CAI Center 

is a part of the Data Management System (DMS) programming available 

to all users of the center. However, because ISC’S has made a unique 

use of CAI, that of curriculum evaluation, only one of the DMS 

programs has been utilized effectively by ISCS. The ISCS staff 

lias designed five programs which manipulate, reduce, and print out 

the data in a format that is utilized effectively by curriculum 

revision teams. Each of the programs in the ISCS data reduction and 
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presentation package interfaces with and is dependent on the CAI 
Center's Dr-©. For input, the ISO’S programs utilize magnetic tapes 
which contain merged and sorted data records produced by the DMS 
system. All the ISCS-DMS analyses are done on the 1500 system. 

The usual mode (and most useful for purposes of revision) of data 
presentation is printed listings, but some data can be obtained 
in the form of punched cards. Each of the analysis programs is 
briefly described below. Complete listings of the ISC’S programs 
are given in ISC'S Tech Memo 1 which is available upon request. 
PROGRAM ISCSA: 

All questions a curriculum revision team may ask cannot be 
anticipated in advance. A detailed listing or account of every 
student response can be used as an aid in answering questions which 
may arise at a later date. 

A detailed listing is available from the DMS. However, because 
of the demand of other users of the CAI system, this listing pro- 
vides all possible data from every student record in a format which 
allows only about 10 responses per page. 

Program ISCSA provides a detailed listing by student, but in 
edited form, as output. Only that portion of the student record 
which could be of use to a revision team is printed. This consists 
of the student number, the question enter and process identifier 
(EPID) , the response identifier (RS), the date of response, latency, 
and the response itself, if it is a keyboard entry. The printed 
output from this program is approximately one-fifth that of the 
DMS Version in terms of the volume of paper produced. Figure A 
illustrates the output from program ISCSA. 



Student 








Latency 


Keyboard 


No. 


LPID 


RS 


DATE 


(min. ) 


Responses 


Z756 


AAEWMA0001 


TM 


09-24-68 


0.46 




Z756 


AAEWMA0002 


TM 


09-24-68 


0.20 




Z756 


AAEXMZ0103 


RV 


09-24-68 


0.78 




Z756 


AAEWMA0002 


TM 


09-24-68 


0.09 




Z756 


AAEXMZ0103 


DC 


09-24-68 


2.00 


BATTERYS 


Z756 


AAEYMA0004 


TM 


09-24-03 


0.34 




Z756 


AAEVMT0105 


RV 


09-25-68 


9.83 




Z756 


AAEYMA0004 


TM 


09-25-68 


0.10 




Z756 


AAEVMTO105 


1T4 


09-25-68 


0.11 




Z756 


AEEFMX0206 


CA 


09-25-68 


1.27 


. 


Z756 


AEEXM50307 


TM 


09-25-68 


2.78 




Z756 


ALEYMA0008 


TM 


09-25-68 


0.18 




Z756 


AFEOMY0409 


TM 


09-25-68 


4.89 




Z756 


AF5VMA0010 


TM 


09-25-68 


0.34 




Z756 


AF5VMA0011 


TM 


09-25-68 


0.17 




Z756 


AFEYMA0012 


TM 


09 25-68 


0.06 




Z756 


AFEXMS0513 


or 


09.-25-68 


16.66 




Z756 


AFEXMS0513 


TM 


09-25-68 


3.48 




Z756 


AEEXMA0014 


TM 


09-25-68 


0.16 




Z756 


AFEYMA0015 


TM 


09-25-68 


0.09 




Z756 


AFKVMT0516 


TM 


09-25-68 


0.28 




Z756 


AGEFMX0617 


VJF 


09-25-68 


4.14 


THE 


Z756 


AGEFMXO718 


RV 


09-25-68 


0.04 




Z756 


AGEFMX0617 


CF 


09-25-68 


1.21 


STARTED TO 
BUBBLE 



FIGURE A 
ISCSA PRINTOUT 



PROGRAM ISCSP : 

This program provides a plot of student progress with respect 
to time. A graph is plotted for each student, with the frame 
number as the abscissa, and the time in minutes as the ordinate. 

A plot from Chapter 1, grade 1 3 is shown in Figure B. 
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FIGURE B 
ISCSP PRINTOUT 

A vertical plot indicates areas in which the student is spending 
most of his time. By scanning the learning materials at this point, 
it can be determined if this was expected, such as a laboratory 
activity, or if it was due to a question, which the student had 
difficulty answering. If the student does finally arrive at a 
correct answer, error analysis done would not indicate the student 
had difficulty with the question. 

A horizontal plot identifies frames through which the student 
proceeded rapidly. A coinciding low error rate may indicate the 
sequence was too easy, whereas a high error rate might suggest the 
student guessed at the response with little thought or had a major 



misconception of the content. If there is a high error rate coupled 
with rapid progress for a series of questions based on laboratory 
activities it is possible that the activities are not doing the 
job well. It is also possible that the student did not complete 
the laboratory work and copied his results from a neighbor. For 
assistance in interpreting the pattern described above, the revision 
team member or evaluator might turn to the type of analysis output 
described next. 

PROGRAM ISCST: 

The program provides a trace of a student’s progress with re- 
spect to response number. The frame number is plotted along the 
abscissa and the number of the response along the ordinate a sample 
trace is shown in Figure C. 




FRAME NUMBER 



123^567890123^67890123^567890 



111111111122222222223 



0 

1 

2 

3 

*1 
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6 
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FIGURE C 
ISCST PRINTOUT 



The data points are identified as the following type of response 
C = correct 



W = incorrect 



U = unrecognizable 
0 or T = latency only 



H = renew 



For a student constantly progressing forward, the slope of 
this trace is - 1. Departures from a slope of - 1 identify possible 
trouble spots in the instructional materials. ISCST is designed 
to flag and interpret two specific response patterns as are depicted 
in Figures D and E. 
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. FIGURE D FIGURE E 

ISCST ZIGZAG TRACE ISCST UNRECOGNIZABLE RESPONSE TRACE 



A zigzag trace, as in Figure D, indicates areas in which the 
student reviewed, proceeded forward, and reviewed again, perhaps 
in order to gain enough information to proceed. This could be 
caused by insufficient information in the instructional materials 
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at that point or the inability of the student to understand the 
material. 

A vertical trace of U's, as in Figure E, signifies unrecognizable 
responses by the student as diagnosed by the computer. This infor- 
mation is of great value in checking student progress if the 
monitoring teacher receives it daily. The teacher can identify 
those students with problems immediately , and determine whether 
difficulties are due to improper use of the C'AI system by the stu- 
dent 3 or if there has been a legitimate response which the computer 
does not accept. 

PROGRAM ISCSB : 

In programming free response questions, which are characteristic 
of much of the ISCS text material, an inherent difficulty is how 
to anticipate all possible correct student; responses. The student 
may type acceptable answers , which are not among those anticipated 
by the programmer. Unfortunately, these are judged incorrect by 
the computer. ISCSB is designed to pick only free response questions 
from the sorted master file and to print them as shown in Figure F. 
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CHAPTER 3 
PAGE 5 

QUESTION 16 



Z751 


WF 


SPEED SHAPE 


Z75I 


WF 


SHAPE AND SPEED 


Z752 


WF 


SHAPE, MOVEMENT 


Z753 


WF 


SPEED DISTANCE 


ZfSH 


WF 


SHAPE, SPEED 


Z755 


WF 


DISTANCE, SPEED 


Z756 


WF 


THE SHAPE AND DISTANCE 


Z756 


WF 


THE SHAPE AND SPEED 


Z756 


WF 


THE SHAPE AND SPEED 


Z757 


CF 


SHAPE AND MOVEMENT 


Z757 


CF 


SHAPE AND MOVEMENT 


Z758 


WF 


DISTANCE, SPEED 



IDENTIFIER 

CEBWMX1637 



FIGURE F 
ISCSB PRINTOUT 

A printout of this type serves two functions. First, acceptable 
but unanticipated answers may be detected, and their error status 
changed. Second, the incorrect answers may indicate the nature 
of the errors and the misconceptions that students have. This 
information serves well in the revision of the instructional materials. 
In the example illustrated in Figure F above, it is apparent that 
student Z 752 correctly answered the question, "Tie amount of change 

in or change in of some object can tell me how much 

force is being exerted." However , the programmer’s apparently 
required "and“ as part of the answer — an error on their part. 
Furthermore, if speed and distance are not acceptable as indicated, 
why do so many student say speed is part of the answer? Perhaps 
the materials leading up to this point need to be examined in order 
to determine possible causes of such a large percentage of wrong 



o 



answers . 
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PROGRAM ISCSM: 

Since its inception , data from the CAI trial have been presented 
in a variety of formats to the project staff. It has been found 
that for revision purposes, a matrix of responses has been the most 
useful format. 

Figure G shows this type of output. The student numbers are 
listed down the left side of the page and the frame identifiers 
(EPID's) across the top. Two columns of response identifiers occur 
under the EPID. The first column under the EPID represents the 
first pass answer of the student. Tie second column under the EPID 
represents the student's last pass answer, if he reviewed back to 
that question and reanswered it. If this second column is blank, 
the student made only one pass and did not review. 

ISCS8 

MATCH IDENTIFIER 



students 


DIOLMX1838 


DIEPMX1939 


DJEQMX20i4l 


DJELIv^S^ 


Z801 


CA 


CA 


WN 


CF 


Z 802 


Ok 


WA 


CA 


VJF 


Z 803 


m 


CA 


CA 


CF 


Z80i) 


Ok 


WA 


CA 


WF 


Z805 


Ok 


WA 


Ck 


CF 


Z 806 


Ok Ok 


WA WA 


WN CA 


CF 


Z807 


Ck 


CA 


CA 


CF 


Z 808 


ww 


CA 


CA WN 


WF 


Z809 


Ck 


WC 


CA 


WF 


Z 810 


Ck 


WA 


WN WN 


CF 


Z 811 


Ok 


Ok 


CA 


VJF 


Z 812 


m 


WA 


WN 


WF 


Z813 


Ck 


CA 


CA 


CF 


Z 8 li) 


m 


Ok 


Ck 


CF 


Z815 


WN 


Ok 


Ck 


CF 


Z 8 l 6 


CA 


WA 


Ck 


CF 


Z817 


CA 


WC 


Ck 


CF 


Z 818 


WN 


WA 


CA 


WF 


Z 819 


CA 


WC 


CA 


CF 


Z820 


Ok 


WC 


Ck 


CF 


ERROR RATE 


0.30 0.30 


0.60 0.60 


0.20 0.20 


0.35 0.35 



FIGURE G 
ISCSM PRINTOUT 
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PROGRAM ISCSM: 

Since its inception , data from the CAI trial have been presented 
in a variety of formats to the project staff. It has been found 
that for revision purposes, a matrix of responses has been the most 
useful format. 

Figure G shows this type of output. The student numbers are 
listed down the left side of the page and the frame identifiers 
(EPID’s) across the top. Two columns of response identifiers occur 
under the EPID. The first column under the EPID represents the 
first pass answer of the student. Hie second column under the EPID 
represents the student’s last pass answer, if he reviewed back to 
that question and reanswered it. If this second column is blank, 
the student made only one pass and did not review. 

ISCS8 

MATCH IDENTIFIER 



STUDENTS 


DIOLMXI 838 


' DIEPMX19 39 


DJEQMX20.41 


L\JELMX2542 


Z801 


CA 


CA 


WN 




CF 


Z802 


CA 


WA 


CA 




VJF 


Z803 


WN 


CA 


CA 




CF 


Z804 


CA 


WA 


CA 




WF 


Z805 


CA 


WA 


CA 




CF 


Z806 


CA CA 


WA WA 


WN 


CA 


CF 


Z807 


CA 


CA 


CA 




CF 


Z808 


m 


CA 


CA 


WN 


VJF 


Z809 


CA 


WC 


CA 




WF 


Z810 


CA 


WA 


WN 


WN 


CF 


Z811 


CA 


CA 


CA 




VJF 


Z812 


WN 


WA 


WN 




WF 


Z813 


CA 


CA 


CA 




CF 


Z8l4 


WN 


CA 


CA 




CF 


Z815 


WN 


CA 


CA 




CF 


Z8l6 


CA 


WA 


CA 




CF 


Z817 


CA 


WC 


CA 




CF 


Z818 


WN 


WA 


CA 




WF 


Z 819 


CA 


WC 


CA 




CF 


Z820 


CA 


WC 


CA 




CF 


ERROR RATE 0. 30 0.30 


0.60 0.60 


0.20 


0_.20__ 


0.35 0.35 



FIGURE G 
ISCSM PRINTOUT 
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The error status of the response can be determined by observing 
the first letter of the response identifier in each of the columns. 

‘C ,! indicates a correct response, ‘W an incorrect response. Hie 
total error rate for each column appears at the bottom of the 
column. If there is no last pass answer, then for error rate 
analyses the first pass answer is also considered to be the last 
pass answer. Questions with a high error rate can easily be detected 
by a quick look at the matrix, and the reason for the high error 
rate may then be investigated. 

OTHER MATRIX OUTPUT: 

The matrix format allows for easy scanning of all student 
first pass responses to a single question, or an individual student’s 
response to successive questions. Because of this functional 
property, it was decided to format other data in the same way. 

Figure H shows a matrix of latency times. Figure I a matrix of 
the number of reviews on each frame, and Figure J a matrix of the 
date of response for each frame. 



■■ 
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ISCS7 



STUDENTS 

Z751 

Z752 

Z753 

Z754 

Z755 

Z756 

Z757 

Z758 

Z759 

Z 760 

Z 761 

Z 762 

Z 763 

Z 764 

Z765 

Z 766 

Z 767 

Z 768 

Z 769 

Z 770 



AVE LATENCY 



CBBVMT0209 

0.0 

8.4 

3.4 
11.8 

9.9 

0.1 

3.4 
8.8 
8.0 
8.6 
0.5 
4.6 
0.0 

23.4 

0.8 

8.2 

24.6 

6.9 
9.2 

6.5 
7.35 



LATENCY 

CB3XMA0010 

0.0 

0.7 

0.2 

0.2 

0.2 

2.0 

3.8 
0.4 
0.3 

3.9 
0.3 
0.2 
0.0 
1.3 
9.6 
0.1 
0.1 
0.8 

1.9 
0.3 

1.31 



IN MINUTES 
CB3YMA0011 
0.1 
0.1 
0.5 
1.1 
0.4 
1.0 
6.0 
0.1 
0.3 
0.3 
0.2 
0.1 
0.0 
1.5 
0.2 
0.4 
0.2 
0.8 
0.0 
0.5 

0.68 



CCEJMY0312 

0.0 

5.5 
0.7 
0.7 
0.2 
0.8 
4.8 
1.0 
0.4 
0.6 
0.6 

1.4 
0.0 
0.2 

1.6 
0.7 
2.1 

4.5 

1.5 

1.1 

1.41 



FIGURE H 

MATRIX OF STUDENT LATENCY 
BY FRAME 



A quick look at the average latency data in Figure H indicates 
that the first frame ( CBBVMT0209 ) required much more time (7-35 
minutes) to corrplete than did any of the others. The third frame 
(CB3XMA0011) required the least amount of time (0.68 minutes) on 
the average. 

A closer look at the latencies for the first frame reveals 
that this larger time was a general characteristic of most of the 
students at this point in the program. A few students, Z751 and 
Z 763 > for example, had very low latencies, indicating that they 
probably skipped over the frame. Other students, Z764 and Z 767 , 
required extremely long times to complete the frame. Hie frame 



in question instructed the students to complete an activity 
(stretching a rubber band) in the laboratory. Hie latency data 
are quite varied, and one may assume that some students either did 
not do the activities or had done them at some time other than when 
instructed to do so. Other students apparently got so involved 
with or distracted by the activities that they took a very long 
time to conplete the frame, they might have had trouble finding 
the equipment! 

The relatively short latency times for the second and third 
frames are related to the fact that these are both information 
presenting (reading only) frames. The shortest frame in this set 
of four is the third one. The fourth frame directs the student 
to plot one point on a graph in his student text. 

The average latency has proven to be a good estimate of how 
long it takes an average “ student to complete a particular frame. 
Frequently, this type information causes the revision team to 
question seriously whether or not a particular activity or chain 
of frames is worth the time it takes students to complete it. 
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ISO’S 7 

WUMEER OP REVIEWS 



STUEENTS 


CBBWlT0209 


CBBXMA 0010 


CBBYMA0011 


CC3JMY0312 


Z751 


0 


0 


0 


0 


Z752 


0 


1 


0 


0 


Z753 


0 


0 


0 


0 


Z75 i ) 


0 


1 


0 


0 


Z755 


0 


0 


0 


0 


Z756 


0 


0 


6 


1 


Z757 


0 


0 


0 


1 


Z758 


0 


0 


0 


0 


Z759 


0 


1 


0 


0 


Z 760 


0 


2 


0 


0 


1161 


0 


0 


0 


0 


1162 


0 


0 


0 


0 


1163 


1 


2 


15 


2 


Z16H 


0 


1 


0 


1 


1165 


0 


0 


0 


0 


1166 


0 


0 


0 


0 


1161 


1 


0 


0 


0 


1166 


2 


2 


0 


0 


1169 


0 


0 


0 


0 


1110 


0 


0 


0 


0 


TOTAL 


4 


10 


21 


5 






FIGURE I 








MATRIX OP STUDENT REVIEWS BY FRAME 





The matrix shown in Figure I is a valuable aid to formative 
evaluators who are concerned with whether students have to review 
through materials frequently — possibly indicating some lack of 
clarity in the materials. The total of 10 reviews for the second 
frame in the sequence suggests that some revision of the text in 
that frame may be needed. The fact that the reviewing was done by 
several students lends weight to the need for revision. The total 
number of reviews for the third frame was the highest (21) but all 
21 reviews were done by only two students. This may indicate that 
these two students needed special attention at this point in the 
course — either help or control of their tendency to ''play around!" 
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31 DATE OF 


RESPONSE 






STUDENTS 


CA99MA0001 


CAB2MA0002 


CABXMA0003 


CABXMY0104 


Z751 


10-25-68 


10-10-68 


10-10-68 


10- 


-10-68 


Z752 


10-30-68 


10-22-68 


10-22-68 


10- 


-22-68 


Z753 


10-15-68 


10-15-68 


10-15-68 


10- 


-15-68 


Z754 


10-16-68 


10-17-68 


10-17-68 


10- 


-17-68 


Z755 


10-11-68 


10-11-68 


10-11-68 


10- 


-11-68 


Z756 


10-09-68 


10-09-68 


10-09-68 


10- 


-09-68 


Z757 


10-21-68 


10-21-68 


10-21-68 


10- 


-21-68 


Z758 


10-11-68 


10-11-68 


10-11-68 


10- 


-11-68 


Z759 


10-10-68 


10-10-68 


10-10-68 


10- 


-10-68 


Z760 


10-21-68 


10-21-68 


10-21-68 


10- 


-21-68 . 


Z761 


10-11-68 


10-11-68 


10-11-68 


10- 


-11-68 


Z762 


10-10-68 


10-10-68 


10-10-68 


10- 


-10-68 


Z763 


10-11-68. 


' 10-11-68 


10-11-68 


10- 


-11-68: 


Z764 


10-21-68 


10-21-68 


10-21-68 


10- 


-21—68 


Z765 


10-24-68 


10-2A-68 


10-24-68 


10- 


-24-68 


Z766 


10-15-68 


10-15-68 


10-15-68 


10- 


-15-68 


Z767 


10-30-68 


10-30-68 


10-30-68 


10- 


-30-68: 


Z768 


10-18-68 


10-18-68 


10-18-68 


10- 


-18-68 


Z769 


10-10-68 


10-10-68 


10-10-68 


10- 


-10-68 


Z770 


10-22-68 


10-22-68 


10-23-68 


10- 


-23-68 



FIGURE J 

MATRIX OF STUDENT DATE OF RESPONSE 



The date of response matrix gives a quick overview of how far 
apart the students were time-wise at a given point in a specific 
chapter. In the data reproduced in Figure J (chapter 3) the time 
spread on the first frame was from 10-9-68 to 10-30-68, or about 
three weeks. This matrix makes it possible to check quickly the 
approximate progress of the class as a whole or of any individual 
within the class. 



MERJC 












•*» CAI 




%£ 1 ^ 1 ^ 



COURSE ISCS9 012 
STUDENT V128 

DATE E.P.I.D. RESPONSE LATENCY 



01 - 16-70 

01 - 16-70 

01 - 16-70 

01 - 16-70 

01-16-79 

01 - 16-70 

01 - 16-70 

01 - 16-70 



LE023 01HD 
LE023 011-ID 
LE024 01HD 
LE024 01HD 
LE024 01HD 
LE024 01HD 
LF028 01HD 
LF028 QUID 



1ST DAY, NO CLOUD COVER, 2ND DAY, CLOUDY, 
3RD DAY, CLOUDY 4TH DAY, PARTLY 
THE OBSERVATION OF TEMPERATURE DIFFERE 
NCE AND WIND DIRECTION CHANGE COULD HAVE 
TOLD ME THAT A LINE OF TEMPERATURE DIF 
FERENCE WAS 

1ST DAY, CLEAR 2ND DAY PARTLY CLOUDY, 3R 
D DAY, CLOUDY 4TH DAY, 



lilU s 

136.9 

294.7 
98.5 
65.2 
62.9 

131.8 

81.4 



'^Latency in Seconds. 



Qu estion List 

LE023 5-10. List changes in the cloud types as the line approached 

and moved through Fargo: 

LE024 5-11. What observation could have told you on days 1 and 2 

that a line of terrperature difference was approaching 
Fargo? 

LE028 5-13* List changes in the cloud cover in Selma as the line 

of temperature difference passed through: 

FIGURE K 

INDIVIDUAL STUDENT COMMENTS 



A second type of analysis procedure provided a listing of all 
students responses to each question printed in sequence by question. 
This listing was prepared for the revision team charged with evaluat- 
ing ^and revising the unit to which the feedback pertained. This 
procedure provided a means of getting the actual student responses 
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33 



to each open-ended question directly to the revision team in printed 
form. The sanple of this listing shown in Figure L is the response 
list for question 5-10. "List changes in the cloud types as the 
line (tenperature difference) approached and moved through Fargo: 1 ' 



CAI ** FSU 



COURSE ISCS9 
DATE E.P.I.D. 

12-0*1-69 LEO 2 3 01HD 

12-0*1-69 LE023 01HD 



RESPONSE 



NOT CLOUDY THE FIRST DAY, COMPLETELY 
CLOUDY THE SECOND AND THIRD DAY, AND 



11-2*1-69 LE023 01HD STRATOS TO CIRRUS AND 

01-05-70 LE023 0IHD 
01-05-70 LE023 01HD 1ST DAY-CLEAR 
01-05-70 LE023 olHD 2ND DAY-CLOUDY 



01-05-70 LE023 011-ID 
01-05-70 LE023 OlHD 
11-2*1-69 KE023 OlHD 
01-08-70 LE023 01HD 



3RD DAY-CLOUDY 

*ITH DAY-PARTLY CLOUDY 

STRATAS TO CIRRUS TO 



LATENCY 
*18.8 
* 16.2 
26.9 
5.1 
23-7 
30.3 
29.8 
2*1.5 
31. *1 
2.5 



01-08-70 LE023 01HD 
01-08-70 LE023 OH-ID 



DAY1 THE TEMPERATURE IN FARGO SHOULD DECR *11.2 
EASE OVER NIGHT BECAUSE THE WARM AIR IS *13-9 
01-08-70 LE023 01HD APPROT 1 CI*iING THE CLOUDS SHOULD BL CLEAR U 1*1*1. 8 

NTIL THE NEXT DAY DAY2 THE CIOUDS SHOULD 32. *1 



01-08-70 LE023 OH-ID 
01-08-70 LE023 QLHD 

01-08-70 LE023 OlHD 
01-08-70 LE023 01HD 



BE FCGGY IN TO THE NIGHT AIMD THE NEXT DA 39-2 
'Y DAY 3 THE CLOUDS SHOULD BL CLEARNING UP *1*1.7 
AND BE CONTENUALLY CLEAR THROUGH THE *1 *1*1.8 



* :: 'Latency given in seconds. 



FIGURE L 

SUMMARY OF STUDENT COMMENTS 
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SUMMARY 

ISCS in its utilization of CAI for formative evaluation has 

r 

evolved techniques valuable in the preparation of a three-year 
science course for classroom presentation. Experience showed that 
it is feasible to utilize CAI in the evaluation of an activity 
centered course , when: 

1. laboratory space is made available in some nearby area preferably 
next-door to the terminal area. 

2. diagrams , figures, and data tables which are difficult or 
impossible to program are provided to the student in a supplemental 
booklet for his reference and keyed to the CAI program directly. 

3. text materials are replicated as closely as possible for CAI 
presentation . 
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4. using the aid of programming macros. 

Year-long courses can be replicated and presented to class- 
room groups via CAI. The time required to prepare all operating 
CAI versions of a one year long course was approximately: 

3 man-months for programming and preparing companion supple- 
ment book 

1 1/2 man-months for entering a. system 

2 man-months for proofing and debugging 

The presence of a classroom teacher is not only desirable, 
but mandatory for CAI presentation of a laboratory course which 
closely parallels a textbook designed for a classroom in which the 

teacher serves as a resource person. 

Although it is desirable to provide students with a written 
record of their course-work as was done with the IBM 1400 system, 
the advantages offered by the more versatile 1500 system with the 
CRT capability outweigh the lack of a typewritten record for the 
student to keep. 
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For most efficient use by revision teams , the voluminous data 
generated by CAI programs need to be summarized and interpreted 
by evaluation group personnel. It is not reasonable, and probably 
not practical, to expect curriculum writers to reduce vast quantities 
of detailed data before making decisions about how to revise a 
chapter or a page. A number of unique analysis programs were « 
developed for this purpose. 

The use of a EPID code for each frame in a 1500 system program 
provides an efficient means by which data can be identified during 
sorting, analysis, and summarizing procedures. Hie time involved 
in pre-coding each frame for its eventual use is well worth the 
investment. The most time consuming portion of the coding procedure 
is the decision making regarding content and process codes. These 
latter two codes can be omitted where use is not anticipated and 
the EPID still retains such critical information as position in 
sequence, type of question or type response. 

To obtain the greatest value from the detailed records obtained 
from CAI, ISCS found that second (or later) trial materials should 
be programmed for CAI presentation rather than the first year’s 
materials which tend to contain the larger gaps or errors that can 
be detected and corrected readily by evaluation techniques other 
than CAI. 












Appendix A 



-Conparlson of CAI and Field First Year Trial 

Text Materials 



TEX'PBOOK 



CHAPTER V 

"EXPERIMENTING” WITH DRAG. 



You have probably concluded by now that there must be something 
special about the force that you have been calling "drag.' It keeps 
coming up again and again. "Drag" involves what is probably the most 
important force known — the force of friction. It is so important 
that we will devote this entire chapter to studying it. As you study 
friction you will also be learning about one of the most important tools 



of science — the experiment. 

To start our study, you will. need to remove the box from the 
balloon-elevator you used in previous X chapters. Then attach a string to 
the box. Put two metal cubes .‘in.ti't ''ancT place it oh' th’e ^loor 1 / Pull on 
the string until the box just begins to move. How much force is needed 
to start the box moving? 
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CAI TEXT 
CHAPTER 5 

"EXPERMHTING" WITH DRAG 



5.1 qu 

You have probably concluded by now that there must 
be something special about the force that you have 
been calling "drag? It keeps coming up again and 

again. Involves what is probably the most 

Inport ant force known — — the force of friction , 
ca drag 
ty Right 

un Take a look at the previous sentence. Look for a 
four letter word that means the same thing as 
friction. It stands out like a sore thumb. 

8i n 

Type it. 
un Type drag 

5.2 gu Friction is so important that we will devote the 

entire chapter to study iti As you study friction, 
you will also be learning about one of the most 
important tools cl science — the experiment. 

To start your "tudy, check to see that you have the 
box from the balloon elevator in your equipment. 

Attach a string to the box. Now put two metal cubes 
in it, place i 4 on the table, and pull on the string 
until the box just begins to move. Type the 4 time 
when you finish, 
ca 

5*3 gu How much force was needed to start the box moving? 

(1) 5 Newtons 

(2) 500 Newtons 

(3) Not very much force 
( 5 ) Quite a bit of force 
(5) I can’t tell for sure. 

ca 5 

ty Right. Ihat’s probably the best answer, 
wa 1 
wb 2 

ty How can you say that without measuring it? Try again, 
wa 3 Perhaps it isn’t much compared to that for pulling • 
an elephant but it’s a lot conpared to pulling a flea. 
Try again, 
wa *1 

It’s quite a bit compared to pulling a flea but 
not much compared to the force needed to pull an 
elephant. Try again, 
un Type either l,2,3,or 4. Try again. 






TEXTBOOK 



Here again is the same question you have not 
been able to answer up to this point. But now your force-measurer gives 
you a way to find the answer. 

-.VM • '• . ••• • . . ; 

Set up a force-measuter as shown in Figure 33. Find a classmate 

• ■. • 

‘ i 4 . ;» • * . 

to help you for the next few minutes.- 




i- • Figure 33 . 

Be sure the card labeled* "newtons" is clipped in plate, and that the metal 

4 1 • , • * . , * 

blade (the heavy one) is 1 zeroed. Slowly 1 move the force— measurer away 

i - • ■ • •• 

from the box and watch the metal blade. Have your partner -tell you when 
the box just - begins to move. Read the force-measurer scale at this moment. 

89. How much force was needed to just start the box moving? 

newtons 



Repeat this procedure several more times to be sure your first reading 
was correct ; t . > • :••• •; •• 

.. ' I !'•'•!*• • . • . » 

90. What was the largest force you could apply before the box just 
started to move;? : . newtons 

; | ; > • ■ > • • ’ . i ' i 
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CAI.TEXT 



5.4 qu This again is the same question you have not been 

able to answer up to this point. But now you might 
have something that will help you get an answer. 

What is it? 

(1) The proctor 

(2) The balloon elevator 

(3) The double pan balance 

(4) The force-measurer 

(5) None will help 
ca 4 

ty Right 
wa 1 

ty The proctor will help when needed but he can’t measure 
the force any better than you can. Try again, 
wb 2 

ty There isn’t a force measuring scale on it. And 

besides, remember all the variables in using it. Try 
again, 
wb 3 

ty You are getting close, but there is something better 
to use with the box. Try again, 
wb 5 

ty Oh yes there is. Try again, 
un Type the number of the answer only 

5.5 qu If the slide projector is not on, turn it on and 

go to slide number 42. Figure 1. 

Type the time when you have it displayed. (Note: 
slide number 42 was a 35 mm kodachrome transparency 
of Figure 33 on the opposite page.) 
ca 0 

5.5a qu Good, check again to see that it is Figure 1. Now 
set up your force-measurer as shown in the slide. 

Is the card labeled "newtons" clipped in place and 
the metal blade (the heavy one) zeroed? (Type .^yes 
or no ) 
ca yes 

ty O.K. Let’s get to work then . Call the proctor, 
wa No 

ty Get it ready then and type yes when the card is 
in place and the blade is zeroed. Then call the 
proctor. 

un Just type either yes or no. 

5.6 qu You will slowly move the force-measurer away from 

the box and watch the metal blade. The proctor 
will tell you when the box just begins to move. 

Read the force-measurer scale at that moment. In 
Section 5.1 in your text record how much force wa3 
needed to just start the box moving. Follow the 
instructions in the text and be sure to answer 
both questions. Type in the time when you finish, 
ca 
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91. Why 'did r:the; box-'-not begin to move ras soon as any force was 



applied to it? 



’ ' • t 

Something prevented the box from moving even though. S' force was present, 
This something has to be called a force under the definitions we have 

■■ y * 

been using. We have already found in a previous chapter" tha:t forces 
cause change - -iri mot ion . 
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CAI TEXT 



Turn off the slide projector. (5 sec. pause) 
qu VJhy did the box not begin to move as soon as you started 
to apply a force on it? 

(1) Something prevented it form moving even though a 
force was present. 

(2) There was a drag so the force applied had to overcome 
the drag before the box could move. 

(3) The force of friction prevented movement until 
the pulling force was greater than the friction 
force , 

(i|) All of the above are good answers. 

(5) None of the above is a good answer. 

ca 4 
ty Good 
wa 1 

ty True— you picked the best answer, but some others are 
also; possible answers. Type the number 4. 
wa 2 

wb 3 _ 

ty True — but others are correct, too. Type 4. 

wa 5 

ty VJrong, try again. 

un Type one number only 1, 2,3*4, or 5. 

qu In the following 3 questions, fill in the blank by typing 
the number of the correct alternative given here. 

(1) direction 

(2) friction 

(3) motion 

(5) .drag 

(5) a force 

Something prevented the box form moving even though 
w as present. 

ca 5 

ty Correct 
wa 2 

W b H \ 

ty You are too specific. (Hint: Friction or drag is .) 

Try again 
wa 1 
wb 3 

ty No. Try again. 

un Type a number, as directed above. 



qu This "something" has to be colled a force under the 

definitions we have beeni using. We have already found in 

a previous chapter that forces cause change in * 

(Select the answer from the list in the last question 
and type its number. ) 
ca 3 

ty Excellent 
wa 1 
wb2 
wb 4 
wb 5 

ty Wrong, Try again. 

un Type a number 1,2, 3*4, or 5- Try again. 
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1 '• TEXTBOOK*: 



. ” Since the rate of motion of* the box changes, 

! i '• • ■ t‘;v '.a. • * ■ .• •:'■/• •••”;’ ' 

this ''something'! must be a force. We will refer to this force as the 



force of friction . 

In studying the force of friction, you will use one of the 



‘if* 



* * + ‘ . , * 

scientist's most important tools — the experiment. You have already 






used the word ''experiment';' several, times. Until* now you.;, have not thought 
very carefully about what it means. You now have the opportunity to find 
out what experiments are. You will do this by actually doing an 
experiment that is designed to tell you. more about the force .of friction. 



1 i * « 
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* \ . * 



Imagine that you have an empty red wastebasket, and a green 



wastebasket full of wooden blocks in front of you on the floor. 



92iV,..-rWhlch' basket do-you think would be' th'e most- difficult to slide 

. :* • .* v ‘i 

along the floor? (That is, which would have more force of 



friction acting on it?) 
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CAI TEXT 



5.10 qu Since the rate of motion of the box changes, this "something" 

must be a force. We refer to this force as the force of 

(Select your answer fhcm the previous list.) 
ca 2 

ty Very good 
wa 3. 

3 

5 

ty No, try again, 
wa 4 

ty There is a better choice. Try again. 

un Type a number for the choice you want, as directed above. 

5.11 qu In studying the force of friction, you will use one of the 

scientist * s most Important tools — the . 

(1) atomic bomb 

(2) hammer 

(3) elephant gun 

(4) experiment 
ca 4 

ty Sure 
wa 1 

ty Boom, You flunk! Try again, 
wa 2 

ty Ouch — pick something better 
wa 3 

ty You ought to be shot. Try again, 
un Pick a number and type it. 

5.12 qu You have already used the word "experiment” several times. 

Until now you have not thought very carefully about what it 
means. You now have the opportunity to find out what experi- 
ments are. You will do this by actually doing an experiment 
that is designed to tell you more about the force of friction. 

Imagine that you have an empty red wastebasket and a green 
wastebasket full of wooden blocks out on the cement sidewalk. 



Which basket do you think would be the most difficult to slide 
along the sidewalk? (That is, which would have more force of 
friction acting on it?) 

Read All the answers before picking the best one. 

(1) the red wastebasket 

(2) the green wastebasket 

(3) the heavier one 

(4) the one with the rougher bottom 
ca 2 

cb 3 
cb 4 

ty Any one of the answers 2, 3, and 4 could be correct. Answer 
2 really isn’t really that good though, since we know color 
isn’t related to friction. We did not have to experiment to 
find this out. 
wa 1 

ty Try again after rereading the question. 

un Type the number of the correct answer. Try again. 
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You probably answered the last question by saying "the heavier 

one/’ ^However, you might have said "the one with- the- rougher bottom." 

You might eyen have, paid "the red one,." but most people would easily see 

. 

• t • * 

that color- isn’t related to friction. They do no ty have- to experiment to 






find this out. 

Scientists experiment when they are not sure which variables are 
hhe important ones. Even when they find it necessary to experiment, they 
often guess ahead of time at the result. Such a guess is sometimes called 

• .• . , ' . • . , y 

hypothesis . If your guess was a good one, your hypothesis : -was that 

* < •*£ ’* ^ it*"* }" < 

"the weight of the blocks was what' made one.' basket , more -dif fijcul-t; to 



slide than the other. " 
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•••As yoii‘ will 1 ‘soon see, knowing the variables in a .situation is 
veryv import art' th in ,! sefc ting : (ip an Experiment." An hypothesis v because it 
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helps identify variables, can be very important. We will*. have more to 
say. about the value -of • hypotheses 1 ‘(the plural form of "hypothesis") later. 
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93 . De scxib 6V how -‘-y dil • wbu Td- find out if your ; 'hypothepis;?about the 
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weight of the bldcks was : correct ? 
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CAI TEXT 



5.13 qu Scientists experiment when they are not sure which variables 
are the important ones. Even when they find it necessary to 
experiment, they often guess ahead of time at the results. 
Such a guess is sometimes called an hypothesis . 

’’The weight of the blocks was what made one basket more dif - 
ficult to slide than the other," is an example of a good 



(1) experiment 

(2) variable 

(3) hypothesis 

ca 3 
ty Good 
wa 1 

ty No. The sentence tells about a thought, not an action. Try 
again, 
wa 2 

ty No — the sentence describes a thought about variables but is 
not a variable itself. Try again, 
un Type either 1, 2, or 3* 

5.14 qu As you will soon see, knowing the variables in a situation is 
very important in setting up an experiment. An hypothesis, 
because it helps identify variables, can be very important. We 
will have more to say about the value of hypotheses (the plural 
form of "hypothesis") later. 

How would you find out if the hypothesis about the weight of 
the blocks was correct? 

(1) think about the problem and predict an answer. 

(2) try it out to see 

(3) ask for the opinions of others. 

ca 2 

ty Right — design an experiment and try it. 
wa 1 
wb 3 

ty Try again — hint — experiment . 
un Type 1, 2, or 3. 
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The CAI Text and Responses as Seen by the Student 



’'EXPERIMENTING” WITH DRAG 



You have probably concluded by now that there must be something special 
about the force that you have been calling "drag,''* It keeps coming up 
again and again. — involves what is probably the most important force 
known — the force of friction . 

Take a look at the previous sentence. Look for a four letter word that 
means the same thing as friction. It stands out like a sore thumb. — 
Type it. 

drag 

Right . 

Friction is so important that we will devote the entire chapter to study 
it. As you study friction you will also be learning about one of the 
most important tools of science — the experiment. 

To start your study, check to see that you have the box from the balloon 
elevator in your equipment. Attach a string to the box. Now put two 
metal cubes in it, place it. on the table, and. pull on the string until the 
box just begins to move. Type the time when you finish. 

411 

How much force was needed to start the box moving? 

1. 5 newtons 

2 . 500 newtons 

3. not very much force 

4. quite a bit of force 

5. I can’t tell for sure 

3 

Perhaps it isn’t much compared to that for pulling an elephant but it’s 
a lot compared to pulling a flea. Try again. 

4 

It’s quite a bit compared to pulling a flea but not much compared to the 
force needed to pull an elephant. Try again. 

5 

Right. That’s probably the best answer. 



